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Abstract We study hierarchical segmentation in the 
framework of edge-weighted graphs. We define ultra- 
metric watersheds as topological watersheds null on the 
minima. We prove that there exists a bijection between 
the set of ultrametric watersheds and the set of hier- 
archical segmentations. We end this paper by showing 
how to use the proposed framework in practice on the 
example of constrained connectivity; in particular it al- 
lows to compute such a hierarchy following a classical 
watershed-based morphological scheme, which provides 
an efficient algorithm to compute the whole hierarchy. 



Introduction 

This papei0 is a contribution to a theory of hierar- 
chical (image) segmentation in the framework of edge- 
weighted graphs. Image segmentation is a process of 
decomposing an image into regions which are homo- 
geneous according to some criteria. Intuitively, a hier- 
archical segmentation represents an image at different 
resolution levels. 

In this paper, we introduce a subclass of edge- weigh- 
ted graphs that we call ultrametric watersheds. Theo- 
rem [T3] states that there exists a one-to-one correspon- 
dence, also called a bijection, between the set of in- 
dexed hierarchical segmentations and the set of ultra- 
metric watersheds. In other words, to any hierarchical 
segmentation (whatever the way the hierarchy is built), 
it is possible to associate a representation of that hierar- 
chy by an ultrametric watershed. Conversely, from any 
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ultrametric watershed, one can infer a indexed hierar- 
chical segmentation. 

This theorem is illustrated on Fig. [TJ that is pro- 
duced using the method proposed in [T] : what is usually 
done is to compute from an original image (Fig. [TJa) a 
hierarchical segmentation that can be represented by 
a dendrogram (Fig. Q]b, see section!?]). The borders of 
the segmentations extracted from the hierarchy (such 
as the one seen in Fig. [TJc) can be stacked to form a 
map (Fig. [T]d) that allows for the visual representation 
of the hierarchical segmentation. Theorem [T3] gives a 
characterization of the class of maps (called ultramet- 
ric watersheds) that represent a hierarchical segmen- 
tation; more surprisingly, Theorem IT3l also states that 
the dendrogram can be obtained after the ultrametric 
watershed has been computed. 

Following [2], we can say that, independently of its 
theoretical interest, such a bijection theorem is useful 
in practice. Any hierarchical segmentation problem is a 
priori heterogeneous: assign to an edge-weighted graph 
an indexed hierarchy. Theorem [T3] allows such classifi- 
cation problem to become homogeneous: assign to an 
edge-weighted graph a particular edge-weighted graph 
called ultrametric watershed. Thus, Theorem [13] gives a 
meaning to questions like: which hierarchy is the closest 
to a given edge- weighted graph with respect to a given 
measure or distance? 

The paper is organised as follow. Related works are 
examined in section [T] We introduce segmentation on 
edges in section [21 and in section [3] we adapt the topo- 
logical watershed framework from the framework of gra- 
phs with discrete weights on the nodes to the one of 
graphs with real-valued weights on the edges. We then 
define (section^) hierarchies and ultrametric distances. 
In section [S] we introduce hierarchical segmentations 
and ultrametric watersheds, the main result being the 



2 



Laurent Najman 



O.G76109 
0,591595 
C.5070B2 
0,422558 
0,338054 
0,253541 



0,15902? 




hierarchical segmentation 

Fig. 1 An example of a hierarchical segmentation produced by the method of L. Guigues £Q e* a '-- The classical order for 
reading the images is (a), (b), (c), (d). But Theorem 1 131 states that the reading order can also be (a), (d), (c), (b) (see text). 



existence of a bijection between these two sets (Th. [T5)) . 
In the last part of the paper (section [5J , we show how 
the proposed framework can be used in practice. After 
proposing (section 16. ip a convenient way to represent 
hierarchies as a discrete image, we demonstrate, using 
ultrametric watersheds, how to compute constrained 
connectivity [3] as a classical watershed-based morpho- 
logical scheme; in particular, it allows us to provide an 
efficient algorithm to compute the whole constrained- 
connectivity hierarchy. 

Apart when otherwise mcntionned, and to the best 
of the author's knowledge, all the properties and theo- 
rems formally stated in this paper are new. This paper 
is an extended version of [4] . 

1 Related works 

This section positions the proposed approach with re- 
spect to what has been done in various different fields. 



When reading the paper for the first time, it can be 
skipped. Readers with a background in classification 
will be interested in section 11.11 those with a back- 
ground in hierarchical image clustering section 11.21 and 
those with a background in mathematical morphology 
by section [T31 



1.1 Hierarchical clustering 

From its beginning in image processing, hierarchical 
segmentation has been thought of as a particular in- 
stance of hierarchical classification [5] . One of the fun- 
damental theorems for hierarchical clustering states that 
there exists a one-to-one correspondence between the 
set of indexed hierarchical classification and a particu- 
lar subset of dissimilarity measures called ultrametric 
distances; This theorem is generally attributed to John- 
son [5J, Jardine et al. H[ and Benzecri [5]. Since then, 
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numerous generalisations of that bijection theorem have 
been proposed (see [5] for a recent review). 

Theorem ll3l fsee below) is an extension to hierarchi- 
cal segmentation of this fundamental hierarchical clus- 
tering theorem. Note that the direction of this extension 
is different from what is done classically in hierarchical 
clustering. For example, E. Diday [5] looks for proper 
dissimilarities that are compatible with the underlying 
lattice. An ultrametric watershed F is not a proper dis- 
similarity, i.e. F(x,y) — does not imply that x = y 
(see section [3]). But F is an ultrametric distance (and 
thus a proper dissimilarity) on the set of connected com- 
ponents of {(x, y)\F(x, y) = 0}, those connected com- 
ponents being the regions of a segmentation. 

Another point of view on our extension is the follow- 
ing: some authors assimilate classification and segmen- 
tation. We advocate that there exists a fundamental 
difference: in classification, we work on the complete 
graph, i.e. the underlying connectivity of the image 
(like the four-connectivity) is not used, and some points 
can be put in the same class because for example, their 
coordinates are correlated in some way with their color; 
thus a class is not always connected for the underlying 
graph. In the framework of segmentation, any region of 
any level of a hierarchy of segmentations is connected 
for the underlying graph. In other words, our approach 
yields a constrained classification, the constraint being 
the four-connectivity of the classes, or more generally 
any connection defining a graph (for the notion of con- 
nection and its links with segmentation, see [51ITU].) 

1.2 Hierarchical segmentation 

There exist many methods for building a hierarchical 
segmentation [11) . which can be divided in three classes: 
bottom-up , top-down or split-and-merge. A recent re- 
view of some of those approaches can be found in [3] . A 
useful representation of hierarchical segmentations was 
introduced in |12) under the name of saliency map. This 
representation has been used (under several names) by 
several authors, for example for visualisation purposes [1] 
or for comparing hierarchies [13] . 

In this paper, we show that any saliency map is an 
ultrametric watershed, and conversely. 

1.3 Watersheds 

For bottom-up approaches, a generic way to build a 
hierarchical segmentation is to start from an initial seg- 
mentation and progressively merge regions together [14] . 
Often, this initial segmentation is obtained through a 
watershed [T21[TS][IB]- See [T7] for a recent review of 



these notions in the context of mathematical morphol- 
ogy HS|. 

Among many others [TH] , topological watershed [2U] 
is an original approach to watersheding that modifies 
a map (e.g., a grayscale image) while preserving the 
connectivity of each lower cross-section. It has been 
proved [IHHH] that this approach is the only one that 
preserves altitudes of the passes (named connection val- 
ues in this paper) between regions of the segmenta- 
tion. Pass altitudes are fundamental for hierarchical 
schemes [T5]. On the other hand, topological watersheds 
may be thick. A study of the properties of different 
kinds of graphs with respect to the thinness of water- 
sheds can be found in (22j[23] . An useful framework 
is that of edge- weighted graphs, where watersheds are 
de facto thin (i.e. of thickness 1); furthermore, in that 
framework, a subclass of topological watersheds satis- 
fies both the drop of water principle and a property 
of global optimality [21] . In this subclass of topological 
watersheds, some of them can be seen as the limit, when 
the power of the weights tends to infinity for some spe- 
cific energy function, of classical algorithms like graph 
cuts or random walkers [25l[26] . 

In this paper, we translate topological watersheds 
from the framework of vertice-weigthed-graphs to the 
one of edge- weighted graphs, and we identify ultra- 
metric watersheds, a subclass of topological watersheds 
that is convenient for hierarchical segmentation. 

2 Segmentation on edges 

This paper is settled in the framework of edge- weighted 
graphs. Following the notations of [27], we present some 
basic definitions to handle such kind of graphs. 

2.1 Basic notions 

We define a graph as a pair X = (V,E) where V is a 
finite set and E is composed of unordered pairs of V, 
i.e., E is a subset of {{x,y} C V | x ^ y}. We denote 
by \V\ the cardinal of V, i.e, the number of elements 
of V. Each element of V is called a vertex or a point 
(of X), and each element of E is called an edge (of X). 
If V ^ 0, we say that X is non-empty. 
As several graphs are considered in this paper, when- 
ever this is necessary, we denote by V(X) and by E(X) 
the vertex and edge set of a graph X . 
A graph X is said complete if E = V(X) x V(X). 
Let X be a graph. If u = {x, y} is an edge of X, we say 
that x and y are adjacent (for X ). Let 7r = (xq, . . . , xg) 
be an ordered sequence of vertices of X, -k is a path 
from xq to xi in X (orinV) if for any i G [1,£\, Xi is 
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adjacent to In this case, we say that xo and Xi 

are linked for X. We say that X is connected if any 
two vertices of X are linked for X. 
Let X and Y be two graphs. If V(Y) C V{X) and E(Y) 
C E(X), we say that V is a subgraph of X and we 
write FCX. We say that Y is a connected component 
of X, or simply a component of X , if y is a connected 
subgraph of X which is maximal for this property, i. e. , 
for any connected graph Z, Y C Z C X implies Z = Y. 
Let X be a graph, and let 5 C E(X). The graph in- 
duced by S is the graph whose edge set is S 1 and whose 
vertex set is made of all points that belong to an edge 
in S, i.e., {{x G V(X) | 3u e 5, i G «}, 5). 

Important remark. Throughout this paper G = 
(V, S) denotes a connected graph, and the letter V (resp. 
£JJ lui/Z always refer to the vertex set (resp. £/ie edge set) 
of G. We will also assume that E =/=9. 
Let S C E. In the following, when no confusion may 
occur, the graph induced by S is also denoted by S. 

Typically, in applications to image segmentation, V 
is the set of picture elements (pixels) and E is any of the 
usual adjacency relations, e.g., the 4- or 8-adjacency in 
2D [25]. 

If S C E, we denote by S the complementary set 
of S in E, i.e., S — E \ S. 



2.2 Segmentation in edge-weigthed graphs 

A deep insight on our work is that we are working with 
edges and not with points: the minimal unit which we 
want to modify is an edge. Indeed, what we need is a 
discrete space in which we can draw the border of a seg- 
mentation, so that we can represent that segmentation 
by its border; in other words, we want to be able to 
obtain the regions from their borders, and conversely. 
In that context, a desirable property is that the regions 
of the segmentation are the connected components of 
the complement of the border. 

As illustrated in Fig. [5Jb, this is not possible to 
achieve with the classical definition of a point-cut. In- 
deed, recall that a partition of V is a collection (V*) of 
non-empty subsets of V such that any element of V is 
exactly in one of these subsets, and that a point-cut is 
the set of edges crossing a partition. Even if we add the 
hypothesis that any (Vi,(Vi x Vi) Pi E) is a connected 
graph, a Vi can be reduced to an isolated vertice, as the 
circled grey-point of Fig. [2Jb. In that case, the comple- 
ment of the point-cut, being a set of edges, does not 
contain that isolated vertice. The correct space to work 
with is the one of edges, and this motivates the follow- 
ing definitions. 



Definition 1 A set C C E is an (edge-)cut (of G) if 
each edge of C is adjacent to two different nonempty 
connected components of C . A graph S is called an seg- 
mentation (of G) if E(S) is a cut. Any connected com- 
ponent of a segmentation S is called a region (of S). 

As mentioned above, the previous definitions of cut 
and segmentation (illustrated on Fig. [SJc) are not the 
usual ones. One can remark the complement of the com- 
plement of a cut is the cut itself, and that any seg- 
mentation gives a partition, the converse being false. 
In particular, Prop. [2Ji below states that there is no 
isolated point in an segmentation. If we need an iso- 
lated point x, it is always possible to replace x with an 
edge {x' , y'}. An application of the framework of hierar- 
chical segmentation to constrained connectivity (where 
isolated points are present) is described in section [5] 

It is interesting to state the definition of a segmen- 
tation from the point of view of vertices of the graph. 
A graph X is said to be spanning (for V) it V(X) = V. 
We denote by <j> the map that associates, to any IcG, 
the graph <f>(X) = {V(X),{{x,y} € E\x G V(X),y G 
V(X)}}. We observe that <f>(X) is maximal among all 
subgraphs of G that are spanning for V(X), it is thus 
a closing on the lattice of subgraphs of G [53] . We call 
(j) the edge-closing. 

Property 2 A graph S C G — (V,E) is a segmenta- 
tion of G if and only if 
(i) The graph induced by E(S) is S; 
(ii) S is spanning for V ; 

(Hi) for any connected component X of S, X — 4>(X). 

Proof Let S be a segmentation of G. Then S is a cut, 
in other word, any edge v — {x, y} g" E(S) is such that 
x an y are in two different connected components of 
S. As G is connected, that implies that S is spanning 
for V. Moreover, E(S) is the set of all edges of S, and 
as S is spanning for G, the graph induced by E(S) is 
(V,E(S)) = S. Let X be a connected component of 
S, suppose that there exists v = {x, y} G E such that 
x and y belong to X and v g 1 E(X). But then v 
E(S) and thus x and y are in two different connected 
components of S, a contradiction. 

Conversely, let S be a graph satisfying (i), (ii) and 
(iii) and let v — {x, y} g" E(S). As, by (ii), S is spanning 
for V, assertion (iii) implies that x and y are in two 
different connected components of E(S). Assertion (i) 
implies that there is no isolated points in S, thus S is 
a cut and thus S is a segmentation of G.D 

2.3 Binary watershed 

Let X be a subgraph of G. We note X + u = (V(X) U 
u,E(X) U {u}). In other words, X + u is the graph 
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Fig. 2 Illustration of segmentation and edge-cut. (a) A graph X. (b) A subgraph of X which is not a segmentation of X: the 
circled grey-point is isolated, and if the point-cut D is the set of dotted-lines edges, D contains only two connected components, 
instead of the expected three (see text), (c) An segmentation of X; the set C of dotted-lines edges is the associated edge-cut 
of X. 



whose vertice-set is composed by the points of V(X) 
and the points of it, and whose edge-set is composed by 
the edges of E(X) and u. An edge u £ E(X) is said 
to be W- simple (for X) (see [50]) if X has the same 
number of connected components as X + u. 
A subgraph X' of G is a thickening (of X) if: 

- X' = X, or if 

— there exists a graph X" which is a thickening of X 
and there exists an edge u W-simple for X" and 
X' = X" + u. 

Thus, informally, a thickening X 1 of X is obtained by 
iteratively adding to AT a sequence of edges u\, . . ■ ,u n , 
i.e. X' = X + u% + . . . + u n , with the constraint that 
in the sequence X$ — X, X n+ \ = X n + u n+ \, the edge 
u n+ \ is W-simple for X n . 

A subgraph X of G such that there does not exist 
a W-simple edge for X is called a binary watershed ( of 
G). 

The following property is a consequence of the def- 
initions of segmentation and binary watershed. 

Property 3 A graph X C G = (V, E) is a segmenta- 
tion of G if and only if X is a binary watershed of G 
and if X is induced by E(X). 



Proof If A is a segmentation, then E(X) is a cut; let 
u £ E(X), u is adjacent to two different non-empty 
connected components of E(X), in other word u is not 
W-simple for X. Thus any segmentation is a binary 
watershed. 

Conversely, let X be a binary watershed, any u g" 
E(X) is not W-simple for X (and thus u is adjacent to 
two different connected components of X). If further- 
more X is induced by E(X) then E(X) is a cut.D 



Thus, starting from a set of edges X, a segmentation 
is obtained by iterative thickening steps until idempo- 
tence. The next section extends the binary watershed 
approach to edge-weighted graphs. 



3 Topological watershed 

3.1 Edge- weighted graphs 

We denote by T the set of all maps from E to M + Given 
any F <E J-, the positive numbers F(u) for u £ E are 
called the weights and the pair (G, F) an edge-weighted 
graph. Whenever no confusion can occur, we will denote 
the edge-weighted graph (G, F) by F. 

For applications to image segmentation, we take for 
weight F(u), where u = {x,y} is an edge between two 
pixels x and y, a dissimilarity measure between x and y 
(e.g., F(u) equals the absolute difference of intensity be- 
tween x and y; see [30] for a more complete discussion 
on different ways to set the map F for image segmen- 
tation). Thus, we suppose that the salient contours are 
located on the highest edges of (G,F). 

Let A S R + and F G F, we define F[X] = {v £ 
E | F(v) < A}. The graph (induced by) F[X] is called a 
(cross) -section of F. A connected component of a sec- 
tion F[X) is called a component of F (at level X). 

We define C(F) as the set composed of all the pairs 
[A, G], where A £ IR + and G is a component of the graph 
F\X\. We call altitude of [A, G] the number A. We note 
that one can reconstruct F from C(F); more precisely, 
we have: 



F{v) = min{A | [A, G] £ C(F), v £ E(C)} 



(1) 



(i 
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For any component G of F, we set h(C) = min{A | [A, G] 
G C(F)}. We define C*(F) as the set composed by all 
[h(C),C] where G is a component of F. The set C*(F), 
called the component tree of F [3TU32] , is a finite subset 
of C(F) that is widely used in practice for image filter- 
ing. Note that the previous equation ([1]) also holds for 
C*(F): 

F(v) = min{A | [A, G] G C*{F), v G E(C)} (2) 

We will make use of the component tree in the proof of 
Pr.[H 

A (regional) minimum of F is a component X of the 
graph F[X] such that for all X 1 < A, F[Xi] H E(X) = 0. 
We remark that a minimum of F is a subgraph of G 
and not a subset of vertices of G; we also remark that 
any minimum X of F is such that |V(X)| > f. 

We denote by M. (F) the graph whose vertex set and 
edge set are, respectively, the union of the vertex sets 
and edge sets of all minima of F. In Fig. [31 boxes are 
drawn around each of the minimum of M (F) . Note that 
A4(F) is induced by E(M.(F)). As a convenient nota- 
tion, and when no confusion can occur, we will some- 
times write X G A4(F) if X is a connected component 
of M(F). 

3.2 Topological watersheds on edge- weighted graphs 

In that section, we extend the definition of topological 
watershed [3U] to edge- weighted graphs, and we give 
an original characterization of topological watersheds 
in that framework (Th. [7]) . 

Let F G T . An edge u such that F(u) = A is said to 
be W- destructible (for F ) with lowest value Ao if there 
exists Ao such that, for all Ai, Ao < Ai < A, u is W- 
simple for F[Xi] and if u is not W-simple for F[Ao]. 

A topological watershed (on G) is a map that con- 
tains no W-destructible edges. 

An illustration of a topological watershed can be 
found in Fig. [3J 

A practical way to obtain a topological watershed 
from any given map is to apply a topological thinning, 
that, informally, consists in lowering W-destructible ed- 
ges. More precisely, a map F' is a topological thinning 
'of F, if: 

- F' = F, or if 

— there exists a map F" which is a topological thin- 
ning of F and there exists an edge u W-destructible 
for F" with lowest value A such that \/v ^ u, F'(v) = 
F"(v) and F'(v) = A , with A < A < F"(v). 

A characterization of a W-destructible edge is pro- 
vided through the connection value. The connection 



value between x G V and y G V is the number 

F(x,y) = min{A | [A, G] G C(F),x G V(C),y € V(C)} 

(3) 

In other words, F(x,y) is the altitude of the lowest 
element [A, G] of C(F) such that x and y belong to G 
(rule of the least common ancestor) . 

In Fig. [3]a and Fig. [3]b, it can be seen that the 
connection value between the points m and p is 6, that 
the one between m and d is 6, and that the one between 
p and d is 5. 

The connection value is a practical way to know if 
an edge is W-destructible. The following property is a 
translation of prop. 2 in |33j to the framework of edge- 
weighted graphs. 

Property 4 (Prop. 2 in [33j) Let F G T. An edge 
v = {x, y} G E is W-destructible for F with lowest 
value X if and only if X = F(x,y) < F(v). 

Two points x and y are separated (for F) if F{x, y) > 
max{Ai, A2}, where Ai (resp. A 2 ) is the altitude of the 
lowest element [Ai,ci] (resp. [A2,C2]) of C(F) such that 
x G ci (resp. y G C2). The points x and y are X-separated 
(for F) if they are separated and A = F(x, y). 

The map F' is a separation of F if, whenever two 
points are A-separated for F, they are A-separated for 
F'. 

If A and Y are two subgraphs of G, we set F(X, Y) = 
min{F(x,y) \ x e X, y e Y}. 

Theorem 5 (Restriction to minima [20j ) Let F 1 < 

F be two elements of F . The map F' is a separation of 
F if and only if for all distinct minima X and Y of 
M{F), we have F'(X,Y) = F{X,Y). 

A graph X is flat (for F) if for all u, v G E(X), 
F(u) = F(v). If X is flat, the altitude of X is the num- 
ber F(X) such that F(X) = F(v) for any v G E(X). 

We say that F is a strong separation of F if F is 
a separation of F and if, for each X 1 G Ai(F'), there 
exists X G M{F) such that X C X' and F(X) = 
F(X'). 

Theorem 6 (strong separation [20j ) Let F and F' 

in T with F' < F. Then F' is a topological thinning of 
F if and only if F' is a strong separation of F. 

In other words, topological thinnings are the only way 
to obtain a watershed that preserves connection values. 

In the framework of edge- weighted graphs, topolog- 
ical watersheds allows for a simple characterization. 

Theorem 7 A map F is a topological watershed if and 
only if: 

(i) M.{F) is a segmentation of G; 
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Fig. 3 Illustration of topological watershed, (a) An edge-weighted graph F. (b) A topological watershed of F. The minima of 
(a) are ({m, i}), ({p, I}), ({g, h}, {c, d}, {g, c}, {h, d}). A box is drawn around each one of the minimum in (a) and (b). 



(ii) for any edge v = {x,y}, if there exist X and Y in 
M(F), X ± Y, such that x G V(X) and y G V{Y), 
then F(v) = F(X,Y). 

Proof Let F be a topological watershed. Thus there 
does not exist any edge W-destructible for F. 

— Suppose that M(F) is not a segmentation of G. 
That means that there exists an edge u = {x, y} G 
E(A4(F)) such that x and y belongs to the same 
connected component X of Ai (F) . That implies that 
F(u) > F(X) = F(x,y). By Pr.H that implies that 
the edge u is W-destructible for F, a contradiction. 
Thus M.(F) is a segmentation of G. 

— As F is a topological watershed, we have by Pr. 0] 
that for any v = {x,y} <E E, F(x,y) — F(v). In 
particular, if there exist X and Y in M(F), X ^ Y, 
such that x € V(X) and y e V(Y), then F(v) = 
F(X,Y). 

Conversely, suppose that F satisfies (i) and (ii). By 
Pr. |U for any edge v = {x,y} € E{M{F)), F(v) = 
F(x,y) — F(X), and thus M(F) does not contain any 
edge W-destructible for F. As, by (i), M(F) is a seg- 
mentation, any edge v £ E(M(F)) satisfies (ii). By 
Pr. |U such an edge v is not W-destructible. Thus F 
contains no W-destructible edge and is a topological 
watershed. □ 

Note that if F is a topological watershed, then for any 
edge v = {x, y} such that there exists X £ M(F) with 
x G V(X) and y G V(X), we have F(v) = F(X). 



4 Hierarchies and ultrametric distances 

Let fl be a finite set. A hierarchy H on Q is a set of 
parts of J? such that 

(i) Q e H 

(ii) for every w G i7, {w} G i? 

(iii) for each pair (h, h') £ H 2 , hnti ^ =S> ft C /i' 
or ft' C ft. 

The (iii) can be expressed by saying that two elements 
of a hierarchy are either disjoint or nested. 

An indexed hierarchy on ft is a pair (H,fi), where 
H denotes a given hierarchy on I? and [i is a positive 
function, defined on and satisfying the following con- 
ditions: 

(i) /x(ft) = if and only if ft is reduced to a singleton 
of 

(ii) if ft C ft', then ^(ft) < ^i(ft'). 



n 

(a) Hierarchy 



n 

(b) Indexed hierarchy 



X 2 



Fig. 4 Hierarchical trees. We have Ai < A3 < A4 and A2 < A4. 

Hierarchy are usually represented using a special 
type of tree called dendrograms (Fig. @}. The leafs of 
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the tree are the data that are to be classified, while the 
branching point (the junctions) are the agglomeration 
of all the data that are below that point. In that sense, 
one can see that, for a given h, ^{h) corresponds to the 
"level" of aggregation, where the elements of h have 
been aggregated for the first time. 

Recall that a dissimilarity on fl is a map d from 
the Cartesian product fl x fl to the set K of real num- 
bers such that: d{LOi,u>2) — d{^2,^i), d{u>i,uji) — and 
d(wi,W2) > for all wi,a;2,W3 G fl. The dissimilarity 
d is said to be proper whenever d(wi,W2) = implies 

A distance d (on fl) is a proper dissimilarity that 
obeys the triangular inequality d{uii,uj2) < d{<jj\,ijj^) + 
d(uj3,LU2) where wi,W2 and W3 are any three points of 
the space. 

The ultrametric inequality [34] is stronger than the 
triangular inequality. An ultrametric distance (on fl) is 
a proper dissimilarity such that, for all wi,u;2,W3 G fl, 
d(wi,w 2 ) < max(d(uJi,uj 3 ),d(u!2,U3)) 

Note that any given partition (fli) of the set fl in- 
duces a large number of trivial ultrametric distances: 

= 0, d(wi, w 2 ) = 1 if wi G J?i, W2 G i 7^ 
and d(wi,w 2 ) = a if i = J, < a < 1. The general 
connection between indexed hierarchies and ultramet- 
ric distances goes back to Benzecri [5] and Johnson [6] . 
They proved there is a bijection between indexed hier- 
archies and ultrametric distances, both defined on the 
same set. Indeed, associated with each indexed hierar- 
chy (if, fj,) on fl is the following ultrametric distance: 

d(uji,uj2) — mm{fj,(h) \h G if, wi G h,u)2 G h}. (4) 

In other words, the distance d(uii,u>2) between two ele- 
ments uj\ and UJ2 in fl is given by the smallest element 
in if which contains both u\ and w 2 - Conversely, each 
ultrametric distance d is associated with one and only 
one indexed hierarchy. 

Observe the similarity between Eq. [H and Eq. [3J 
Indeed, connection value is an ultrametric distance on 
V whenever F > 0. More precisely, we can state the 
following property, whose proof is a simple consequence 
of EqH and Eq. ED 

Property 8 Let F G J '. Then F(X,Y) is an ultra- 
metric distance on M.(F). If furthemore, F > 0, then 
F(x,y) is an ultrametric distance on V. 

Let <F be the application that associates to any F G 
T the map \P(F) such that for any edge {x,y} G E. 
<I r (F)({x,y}) — F(x,y). It is straightforward to see 
that W{F) < F, that &(&(F)) = &(F) and that if 
F' < F, W(F') < <F{F). Thus V is an opening on 
the lattice (J 7 , <) [3S]. We observe that the subset of 
strictly positive maps that are defined on the complete 



graph (V, V x V) and that are open with respect to & 
is the set of ultrametric distances on V. The mapping 
& is known under several names, including "subdomi- 
nant ultrametric" and "ultrametric opening" . It is well 
known that W is associated to the simplest method for 
hierarchical classification called single linkage cluster- 
ing [336], closely related to Kruskal's algorithm [37] 
for computing a minimum spanning tree. 

Thanks to Th. [7J we observe that if F is a topologi- 
cal watershed, then &(F) = F. However, an ultrametric 
distance d may have plateaus, and thus the weighted 
complete graph (V", V xV,d) is not always a topological 
watershed. Nevertheless, those results underline that 
topological watersheds are related to hierarchical clas- 
sification, but not yet to hierarchical segmentation; the 
study of such relations is the subject of the rest of the 
paper. 

5 Hierarchical segmentations, saliency and 
ultrametric watersheds 

Informally, a hierarchical segmentation is a hierarchy of 
connected regions. However, in our framework, if a seg- 
mentation induces a partition, the converse is not true 
(see Pr.[2]); thus, as the union of two disjoint connected 
subgraphs of G is not a connected subgraph of G, the 
formal definition is slightly more involved. 

A hierarchical segmentation (on G) is an indexed hi- 
erarchy (H, n) on the set of regions of a segmentation S 
of G, such that for any h G H, 4>(Ux£hX) is connected 
{(f) being the edge-closing defined in section [5]). 

For any A > 0, we denote by H[X] the graph in- 
duced by {(fj(UxehX)\h G H,fi(h) < A}. The following- 
property is an easy consequence of the definition of a 
hierarchical segmentation. 

Property 9 Let {H, /i) be a hierarchical segmentation. 
Then for any A > 0, the graph H[\] is a segmentation 
ofG. 

Proof Let (iJ, /x) be a hierarchical segmentation, and 
let A > 0. Suppose that H[X] is not a segmentation, i.e. 
that if [A] is not a cut. Then there exists a connected 
component X of if [A] and v = {x,y} G if [A] such that 
x G X and y G X. That implies that 4>{X) =^ X, a 
contradiction with the definition of a hierarchical seg- 
mentation. □ 

Prop. |S] implies that the connection value defines a 
hierarchy on the set of minima of F. If F is a topological 
watershed, then by Th. [Jj M(F) is a segmentation of 
G, and thus from any topological watershed, one can 
infer a hierarchical segmentation. However, F[X] is not 
always a segmentation: if there exists a minimum X of 



On the equivalence between hierarchical segmentations and ultrametric watersheds 



9 



F such that F(X) = A > 0, for any Ai < A , F[Ai] 
contains at least two connected components X\ and 
X 2 such that |V(Xi)| = \V{X 2 )\ = 1. Note that the 
value of F on the minima of F is not related to the 
position of the divide nor to the associated hierarchy of 
minima/segmentations. This leads us to introduce the 
following definition. 

Definition 10 A map F £ T is an ultrametric water- 
shed if F is a topological watershed, and if furthemore, 
for any X £ M{F), F(X) = 0. 

Definition \TU\ directly yields to the nice following prop- 
erty, illustrated in Fig.[5j that states that any level of an 
ultrametric watershed is a segmentation and conversely. 

Property 11 A map F is an ultrametric watershed if 
and only if for all A > 0, F[X] is a segmentation of G. 

Proof Suppose that F is an ultrametric watershed, then 
it is a topological watershed, and by Th. [7] (i) , Ai (F) 
is a segmentation of G. But as the value of F on its 
minima is null, then any cross-section of F is a segmen- 
tation of G. 

Conversely, if for any A > 0, F [A] is a segmen- 
tation of G, then F contains no W-destructible edge 
for F. Indeed, suppose that there exists an edge v W- 
destructible for F, let A = F(v), then v is W-simple for 
F[X\. In other words, adding v to F[X] does not change 
the number of connected components of F[X]. This is 
a contradiction with the definition of a segmentation. 
Hence F is a topological watershed. Furthermore, as 
F[X] is a segmentation for any A > 0, the value of F on 
its minima is null, hence F is an ultrametric watershed. 
□ 

By definition of a hierarchy, two elements of H are 
either disjoint or nested. If furthermore (H, /j,) is a hi- 
erarchical segmentation, the graphs E(H[X)) can be 
stacked to form a map. We call saliency map |12) the 
result of such a stacking, i.e. a saliency map is a map 
F such that there exists (H, /x) a hierarchical segmen- 
tation with F(v) = min{A|u £ E(H[X])}. 

Property 12 A map F is a saliency map if and only 
if F is an ultrametric watershed. 

Proof If F is a saliency map, then there exists (H, /i) a 
hierarchical segmentation such that F(v) = min{A | v £ 
E(H(X))}. But F[X] = {v | F(v) < X} = {v | min{A | v £ 
E{H[X})} <X} = H[X] and thus by Pr.® for any A > 0, 
F[X] is a segmentation. By Pr. [TTJ F is an ultrametric 
watershed. 

Conversely, let F be an ultrametric watershed, and 
let C*(F) be the component tree of F. We build the 



pair (H, fi) in the following way: h £ H if and only if 
there exists [A, C] £ C*{F) such that h = {X t | £ 
Ai(F) and Xj C C}; in that case, we set fj,(h) = X. 

Then (H, /Y) is a hierarchical segmentation. Indeed, 
let h and h! two elements of H such that there exists 
[A, X] and [A', Y] in C*(F) with h = {X , . . . , X p | X t £ 
M(F) and X l C X} and with h! = {Y , . . . ,Y n \ Y £ 
M(F) and Yi C Y}. 

— by Th.0 M(F) is a segmentation, 

— we set A max = max{F(t>) | v £ E}, it easy to see 
that [A max , (V,E)} £ C*(F), thus {U XeM(F) {X}} £ 
H; 

— any minimum X of F is such that [0, X] belongs to 
C*(F), thus {X} £ H; 

— furthermore, h and h' are either disjoint or nested: 

— either disjoint: suppose that X n Y = 0, in that 
case h and h' are also disjoint; 

— or nested: suppose that XDY ^ 0, then as X and 
Y are two connected components of the cross- 
sections of F, either X C Y or Y C X; suppose 
that X C Y; by reordering the Xi and the 5^-, 
that means that X{ = Y^ for i = 0, . . . ,p, p < n. 
In other words, h C b! . 

— by construction, fi(h) = if and only if there exists 
X £ M(F) such that h = {X}; 

— If h C /i', then < n(h'), because in that case, 
X C Y and thus A < A'. 

Thus (H, yu) is a indexed hierarchy on (F) . 

Furthermore, ^(Ux^aAj) is connected: more pre- 
cisely, as Ai(F) is a segmentation, and as AT is a con- 
nected component of the cross-sections of F, we have 
4>(LiXiehXi) = X. Thus (H, fj,) is a hierarchical segmen- 
tation. □ 

The following theorem, a corrolarv of Prop . [T2l states 
the equivalence between hierarchical segmentations and 
ultrametric watersheds. It is the main result of this pa- 
per. 

Theorem 13 There exists a bijection between the set 
of hierarchical segmentations on G and the set of ultra- 
metric watersheds on G. 

Proof By Pr. ll2l any ultrametric watershed is a saliency 
map, thus for any ultrametric watershed, there exists 
an associated hierarchical segmentation. 

Conversely, for any hierarchical segmentation, there 
exists a unique saliency map, thus by Pr. 1121 a unique 
ultrametric watershed. □ 

Th.[l3]states that any hierarchical segmentation can be 
represented by an ultrametric watershed. Such a rep- 
resentation can easily be built by stacking the border 
of the regions of the hierarchy (see Pr. [S] and [T^l but 
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5> 



(a) Ultrametric watershed F 



(b) cross-section of F at level 5 



Fig. 5 An example of an ultrametric watershed F and a cross-section of F. In (a), a box is drawn around each one of the 
minima of F, and in (b), a box is drawn around each one of the connected components of the cross-section of F. Remark that 
the minima of F, as well as any cross-section of F, form a segmentation of the graph. 



also [TJHHEI])- More interestingly, Th. [T2] also states 
that any ultrametric watershed yields a hierarchical seg- 
mentation. As the definition of topological watershed is 
constructive, this is an incentive to searching for algo- 
rithmic schemes that directly compute the whole hier- 
archy. An exemple of such an application of Th. [13] is 
developped in section [51 

As there exists a one-to-one correspondence between 
the set of indexed hierarchies and the set of ultramet- 
ric distances, it is interesting to search if there exists 
a similar property for the set of hierarchical segmenta- 
tions. Let d be the ultrametric distance associated to 
a hierarchical segmentation (H, fi). We call ultrametric 
contour map ( associated to (H, /i) ) the map dE such 
that: 

1. for any edge v G E(H[0]), then d E (v) = 0; 

2. for any edge v = {x, y} G E(H[0]), d E (v) = d(X, Y) 
where X (resp. Y) is the connected component of 
H[0] that contains x (resp. y). 

Property 14 A map F is an ultrametric watershed if 
and only if F is the ultrametric contour map associated 
to a hierarchical segmentation. 

Proof Let F be an ultrametric watershed. By Pr. (5[ 
F(X, Y) is an ultrametric distance on M. (F). By Pr.fTTl 
F is a saliency map, hence there exists a hierarchical 
segmentation (H, /x) such that F(v) = min{A | v G 
E(H[X])}. In particular, 

1. for any edge v G E{H[0]), then F(v) = 0; 



2. for any edge v = {x, y} G E(H[0}), F(v) = F(X, Y) 
where X (resp. Y) is the connected component of 
H[Q] that contains x (resp. y). 

Hence F is an ultrametric contour map associated to a 
hierarchical segmentation. 

Conversely, let d E be an ultrametric contour map 
associated to a hierarchical segmentation (H, /x) . Then 
by Th. [71 d E is a topological watershed. Indeed, as 
H is a hierarchical segmentation, H[0] = A4(d E ) is 
a segmentation of G, and furthermore for any edge 
v = {x, y}, if there exist X and Y in A4(d E ), X ^ Y , 
such that x G V(X) and y G V(Y), then d E {v) = 
d(X,Y)=d E {X,Y). 

Moreover, for any v G M(d E ), d E (v) = 0, hence d E 
is an ultrametric watershed. □ 



6 How to use the ultrametric watershed in 
practice: the example of constrained 
connectivity 

Let us illustrate the usefulness of the proposed frame- 
work by providing an original way of revisiting con- 
strained connectivity hierarchical segmentations [3] , whi- 
ch leads to efficient algorithms. This section is meant as 
an illustration of our framework, and, although it is self- 
sufficient, technical details can be somewhat difficult 
to grasp for someone not familiar with the watershed- 
based segmentation framework of mathematical mor- 
phology |17j . We plan to provide more information in 
an extended version of that section. 
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In this section, we propose to compute an ultra- 
metric watershed that corresponds to the constrained 
connectivity hierarchy of a given image. We show that, 
in the framework of edge-weighted segmentations, con- 
strained connectivity can be thought as a classical mor- 
phological scheme, that consists of: 

— computing a gradient; 

— filtering this gradient by attribute filtering; 

— computing a watershed of the filtered gradient. 

We first discuss how to represent hierarchical segmen- 
tations, then give the formal definition of constrained 
connectivity, and then we move on to using ultrametric 
watersheds for computing such a hierarchy. In the last 
part of the section, we will show some other examples 
related to the classical watershed-based segmentation 
schemes. 

6.1 Representations of hierarchical segmentations 

As we mentionned in section 12.21 one of the motiva- 
tions of this work is to be able to imbed the hierarchi- 
cal segmentation in a discrete space in a way that can 
be represented. Until now, we have used the classical 
representation of a graph for all of our examples. 

For the purpose of visualisation, it is enough to rep- 
resent the image by a grid of double resolution. For 
example, with the usual four connectivity in 2D, each 
pixel will be the center of a 3x3 neighborhood, and if 
two pixels share an edge, the two corresponding neigh- 
borhoods will share 3 elements corresponding to that 
edge. The representation of an ultrametric watershed 
with double resolution can be seen in Fig. [BJa. 
Remark: A convenient interpretation of the doubling 
of the resolution can be given in the framework of cubi- 
cal complexes, that have been popularized in computer 
vision by E. Khalimski [35], but can be found earlier 
in the literature, originally in the work of P.S. Alexan- 
droff [MED]- 

Intuitively, a cubical complexe can be seen as a set 
of elements of various dimensions (cubes, squares, seg- 
ments and points) with specific rules between those el- 
ements. The traditional vision of a numerical image as 
being composed of pixels (elementary squares) in 2D or 
voxels (elementary cubes) in 3D leads to a natural link 
between numerical images and complexes. The repre- 
sentation of an ultrametric watershed in the Khalimski 
grid can be seen in Fig. \5\b. 

The framework of complexes is useful in the study 
of topological properties [41]. It is indeed possible to 
provide a formal treatment of watersheds in complexes, 
which we will not do in this paper. The interested reader 
can have a look at [4"2"] . 
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6.2 Constrained connectivity 

This section is a reminder of P. Soille's approach [3], 
using the same notations. 

Let / be an application from V to R, i.e. an image 
with values on the points. For any set of points t/cy, 
we set 

R f (U) = S up{f(x)-f(y)\x,yeU}. (5) 

The number Rf(U) is called the range of U (for /). 

For any x G V, and for any a > 0, define 03] the 
a-connected component a-CC(x) as the set: 

a-CC(j:) = {x} U {y G V | there exists a path 

7r = {x Q =x,...,x n = y}, 
n > 0, such that 
Rf({xi,x l+ i}) < a, 
for all < i < n} (6) 

An essential property of the a-connected compo- 
nents of a point x is that they form an ordered sequence 
(i.e a hierarchy) when increasing the value of a: 

a-CC(x) C j3-CC(x) (7) 

whenever /3 > a. An example of such a hierarchy is 
given in Fig. [7] 

We now define the (a, unconnected component of 
an arbitrary point x as the largest a-connected compo- 
nent of x whose range is lower that cj; more precisely, 

(a,u)-CC(x) = sup{f3-CC(x) \ f3 < a and 

R f ((3-CC(x)) < oj} (8) 

The (a, w)-CCs also define a hierarchy, that is called 
a constrained connectivity hierarchy. We have: 

(a,w)-CC(ar) C (a',u')-CC(x) (9) 

whenever a' > a and Vu/ > u>. In practice [3J, we are 
interested in this hierarchy for a = w, i.e., for any x G V 
and any A > 0, we are looking for (A, A)-CC(x). 

Thus, informally, a hierarchy of a-connected com- 
ponents is given by connectivity relations constrain- 
ing gray-level variations along connected paths; a con- 
strained connectivity hierarchy is given by connectivity 
relations constraining gray-level variations both along 
connected paths and within entire connected compo- 
nents. 

An example of a constrained-connectivity hierarchy 
is given in Fig. [5J 
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(a) Ultrametric watershed F seen with double 
resolution 
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(b) Ultrametric watershed F seen in the Khal- 
imski grid 



Fig. 6 Two possible representations of the ultrametric watershed F of Fig. 0a. 
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Fig. 7 Example from [3| of a 7x7 image and its partitions into a-connected components for a ranging from to 5. (a) 0-CCs. 
(b) 1-CCs. (c) 2-CCs. (d) 3-CCs. (e) 4-CCs. (f) 5-CCs. 



6.3 Ultrametric watershed for constrained connectivity 

In that section, we show how to build a weighted graph 
on which the ultrametric watershed corresponding to 
the hierarchy of constrained connectivity can be com- 
puted. Intuitively, this weighted graph can be seen as 
the gradient of the original image. We compute an ul- 
trametric watershed for the hierarchy of a-connected 
components. We filter that watershed to obtain the 
family of (a, unconnected components. We then show 
how to directly compute the ultrametric watershed cor- 



responding to the hierarchy of (a, unconnected compo- 
nents. 

Constrained connectivity is a hierarchy of flat zones 
of /, in the sense where the 0-connected components of 
/ are the zones of / where the intensity of / does not 
change. In a continuous world, such zones would be the 
ones where the gradient is null, i.e. V/ — 0. However, 
the space we are working with is discrete, and a flat 
zone of / can consist in a single point. In general, it 
is not possible to compute a gradient on the points or 
on the edges such that this gradient is null on the flat 
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Fig. 8 Example from |3] of a 7x7 image and its partitions into (a, w)-connected components using identical values for the 
local and global range parameters ranging from 1 to 6. (a) (1, l)-CCs. (b) (2, 2)-CCs. (c) (3, 3)-CCs. (d) (4, 4)-CCs. (e) (5, 
5)-CCs. (f) (6, 6)-CCs. 



zones. To compute a gradient on the edges such that the 
gradient is null on the flat zones, we need to "double" 
the graph, for example we can do that by doubling the 
number of points of V and adding one edge between 
each new point and the old one (see Fig.^b)). 



of / [44]. It is easy to see that the flat zones of /, i.e. 
the O-connected components of / are (in bijection with) 
the connected components of the set {v — {x,y} G 
E 1 \F({x,y}) = 0}. 




® 

(a) 



Fig. 9 Doubling the graph, (a) Original graph with weights 
/ on the vertices, (b) Double graph, with weigths / on the 
vertices and the gradient F on the edges (see text). 



More precisely, if we denote the points of V by V = 
{xo, ■ ■ ■ , x n }, we set V = {x' Q , . . . , x' n } (with V D V — 
0), and E' = {{x^x^} | < i < n}. We then set 
Vx = VUV' and E x = E U E' . By construction, as G = 
(y, E) is a connected graph, the graph G\ — (Vi 7 Ei) is 
a connected graph. We also extend / to V' , by setting, 
for any x' £ V, f(x') = f(x), where {x,x'} £ E' . 

Let (Vi ,Ei,F) be the weighted graph obtained from 
/ by setting, for any {x,y} G E 1: F({x,y}) = \f(x) - 
f(y)\. The map F can be seen as the "natural gradient" 
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Fig. 10 Doubling the graph as an image, (b) Doubling the 
graph of Fig. [9]a. (b) The gradient of (a) (see text). 



Let us note that it is also possible, for the purpose 
of visualisation, to double the graph as an image, i.e., 
to multiply the size of the image by 2. On the graph of 
Fig.[5]a, that gives the image of Fig.[TUJa. Then the gra- 
dient can be seen as an image (Fig. [TU]b) as described 
in section 16.11 This representation will be adopted in 
all the subsequent figures of the paper. 

Let W 1 be a topological watershed of F. From Th.[7] 
and Eq. [3l if W x ({a;,2/}) = A, there exists a path n = 
{xq = x, . . . , x n — y} linking x to y such that the alti- 
tude of any edge along n is below A, i.e. we have, for 
any < i < n, F({xi,x i+ i}) = \f(xi) - /(afi+i)| < A. 
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(e) Ultrametric watershed for the constrained connectivity 



Fig. 11 Example of a constrained connectivity hierarchy, (a) Original image (the one of Fig. [TJ from 3 . (b) Doubling of (a), 
(c) Gradient of (b). (d) Topological watershed of the gradient, that is the ultrametric watershed W 1 for the a-connectivity 
that, (e) Ultrametric watershed W 2 for the constrained connectivity, (see text) 
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The following property, the proof of which is left to the 
reader, states that the hierarchy of a-connected com- 
ponents is given by W 1 . 

Property 15 We have 

— W 1 is an ultrametric watershed; 

— W 1 is uniquely defined (if W' is a topological wa- 
tershed ofF, then W = W 1 ); 

— let A > and let X be a connected component of the 
cross-section VP rl [A]; then for any x € V(X) \ V' , 
X-CC(x) = V(X)\V. 

Pr. [Tni is illustrated on Fig. [TTJd. Let us stress that 
Fig. [TTJd sums up in one image all the images of Fig. 
One can notice that Rf is increasing on 2 V , i.e. 
Rf(X) C Rf(Y) whenever ICY". Thus Rf is increas- 
ing on C(W 1 ), and by removing the connected compo- 
nents of C(W 1 ) that are below a threshold uj for Rf, 
we have an attribute filtering which is idempotent (the 
values on the points do not change), thus it is a clos- 
ing. More precisely, we denote by (-Ra)a>o) the family 
of maps obtained by applying this closing on W 1 for 
varying A, i.e., for any A > and any {x,y} € E\, we 
set 

R x ({x,y}) = mm{\' | [A', C] € C(W 1 ), 

xeV(C),yeV(C), 
Rf(V(C)) > A} (10) 

In other words, the altitude for R\ of the edge {x, y} 
is the altitude of the lowest component of C(W 1 ) that 
contains both x and y and such that the range of that 
component is greater than A. 

The family (-Ra)a>o allows us to retrieve the (a,uj)- 
CCs of /: surprisingly, it can be shown that any R\ 
is a topological watershed, and thus M(R\) is a seg- 
mentation from which it is easy to extract the (A, A)- 
connected component of a point, as the minimum of 
M(R\) that contains that point (See Pr. 1161 below for 
a more formal setting). 

Moreover, one can directly compute the ultrametric 
watershed associated to the hierarchy of (a, (^-constrai- 
ned connectivity. We set: 

W 2 ({x,y})=mm{R f (V(C)) \ [\,C]&C(W r ), 

x £ V(C), 

yeV(C)} (11) 

In other words, the altitude for W 2 of the edge {x, y} 
is the range of the lowest component of C(W 1 ) that 
contains both x and y. One can remark that Eq. [TT1 
corresponds to Eq.[5]for the framework of segmentation. 

The following property, the proof of which is left to 
the reader, states that the hierarchy of (a, unconnected 
components is given by W 2 . 



Property 16 We have 

— VA > 0. R\ is a topological watershed; 

— VA > 0, W 2 [\] =M(Rx) ; 

— W 2 is an ultrametric watershed; 

— W 2 is uniquely defined; 

— let A > and let X be a connected component of the 
cross-section W 2 [\]; then for any x € V(X) \ V 1 , 
{X,X)-CC(x) = V(X)\V. 

Prop. \Wi illustrated on Fig. HJJe, thus gives an ef- 
ficient algorithm to compute the hierarchy of (a, un- 
constrained connectivity. Indeed, Eq. [TTJ can be com- 
puted in constant time [45] on C(W l ), which itself can 
be computed in quasi- linear time [32]. Such an algo- 
rithm is much faster than the one proposed in [3J, that 
computes only one level of the hierarchy. 

Let us stress that for an algorithmic/implementation 
point of view, it is not necessary in practice to double 
the image. Furthermore, for an efficient computation of 
the hierarchy, a minimum spanning tree or a compo- 
nent tree of the gradient can also be used instead of 
an ultrametric watershed, without changing the overall 
theoretical complexity of the algorithm. But for visu- 
alisation purpose, the ultrametric watershed is neces- 
sary. Moreover, those tools can be combined; indeed, 
one can compute a topological watershed on the graph 
of a minimum spanning tree. In a forthcomming paper, 
we will propose various data structures, including but 
not limited to component tree and minimum spanning 
tree, that allows an efficient computation of hierarchi- 
cal segmentations. We will also study how to extend 
Prop. [16] in order to compute any granulometry of op- 
erators (strong hierarchies in the sense of [9]). 

A example of the application of the properties of 
this section to a real image is given in Fig. |T2j Visual- 
ising W 2 allows to assess some of the qualities of the 
hierarchy of constrained connectivity. One can notice 
in Fig. 1121 c a large number of transition regions (small 
undersirable regions that persist in the hierarchy), and 
this problem is known [46j . As W 2 is an image, a num- 
ber of classical morphological schemes {e.g., area fil- 
tering that produces a hierarchy of regions classified 
according to their size or area; see [17] for more de- 
tails) can be used to remove those transition zones (see 
Fig. [T2]d for an example). Studying the usefulness of 
such schemes is the subject of future research. 



6.4 Links with other hierarchical schemes 

As we have shown, and as stated by Th. [T31 any hi- 
erarchical scheme can be represented by and computed 
through an ultrametric watershed. This is in particu- 
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Fig. 12 Soille's (a, cj)-constrained connectivity hierarchy, (a) Original image, (b) Ultrametric watershed W 1 for the a- 
connectivity. (c) Ultrametric watershed W 2 for the constrained connectivity, (d) Ultrametric watersheds corresponding to 
one of the possible hierarchies of area- filterings on W 2 . 



lar true for the classical watershed-based segmentation 
algorithms. 

Fig. [13] is an illustration of the application of the 
framework developped in this paper to a classical hier- 
archical segmentation scheme based on attribute open- 
ing [TWTTIIBT] ■ The attribute opening tends to produce 
large plateaus where a watershed can be located any- 
where; in particular, the contours at a given level of 
the hierarchy can be choosen differently depending on 
the filtering level, and one has to take care of indeed 
producing a hierarchy. In contrast, the ultrametric wa- 
tershed will always choose a contour that is present at 
a lower level of the hierarchy. 

Fig. [TH shows some of the differences between ap- 
plying an ultrametric-watershed scheme and applying 
a classical watershed-based segmentation scheme, e.g. 
attribute opening followed by a watershed [T5]. As wa- 
tershed algorithms generally place watershed lines in 
the middle of plateaus, the contours produced by the 
classical watershed-based segmentation scheme do not 



lead to a hierarchy, and the two schemes give quite dif- 
ferent results. 




Fig. 14 Zoom on a comparison between two watersheds of 
a filtered version of the image 1131 a. Morphological filtering 
tends to create large plateaus, and both watersheds (a) and 
(b) are possible, but only (a) is a subset of a watershed of ll3l a. 
No hierarchical scheme will ever give a result as (b). 
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(a) Original image 



(b) Ultrametric watershed 



(c) Cross section of (b) 



Fig. 13 Example of ultrametric watershed. On (a), a box is drawn around the zoomed part used in Fig. 1141 



7 Conclusion 

In this paper, we have shown (Th. 1131) that any hier- 
archical segmentation can be represented by an ultra- 
metric watershed, and conversely that any ultrametric 
watershed leads to a hierarchical segmentation. Th. H3l 
thus offers an alternative way of thinking hierarchical 
segmentation that complete existing ones (ultramet- 
ric distances, minimum spanning tree, . . .) We have 
seen how to apply Th. [T3J to directly compute the con- 
strained connectivity hierachy as an ultrametric water- 
shed, leading to a fast algorithm. An important research 
direction is to provide a generalization of this scheme 
for computing any hierachical segmentation. 

As a step in this direction, future work will pro- 
pose novel algorithms (based on the topological wa- 
tershed algorithm [33 ) to compute ultrametric water- 
sheds, with proof of correctness. It is important to note 
that most of the algorithms proposed in the literature 
to compute saliency maps are not correct, often because 
they rely on wrong connection values or because they 
rely on thick watersheds where merging regions is diffi- 
cult [22] . 

On a more theoretical level, this work can be pur- 
sued in several directions. 

— We will study lattices of watersheds [UJ and will 
bring to that framework recent approaches like scale- 
sets [T] and other metric approaches to segmenta- 
tion [15] . For example, scale-sets theory considers a 
rather general formulation of the partitioning prob- 
lem which involves minimizing a two-term-based en- 
ergy, of the form AC + D, where D is a goodness-of- 
fit term and C is a regularization term, and proposes 
an algorithm to compute the hierarchical segmenta- 
tion we obtain by varying the A parameter. As in 
the case of constrained connectivity (see section [5] 



above), we can hope that the topological watershed 
algorithm 33 can be used on a specific energy func- 
tion to directly obtain the hierarchy. 

— Subdominant theory (mentionned at the end of sec- 
tion 2]) links hierarchical classification and optimi- 
sation. In particular, the subdominant ultrametric 
d! of a dissimilarity d is the solution to the following 
optimisation problem for p < oo: 

min{||cZ — d \ \p \ d' is an ultrametric distance 

and d! < d} (12) 

It is certainly of interest to search if topological 
watersheds can be solutions of similar optimisation 
problems. 

— Several generalisations of hierarchical clustering have 
been proposed in the literature [2J. An interesting 
direction of research is to see how to extend in the 
same way the topological watershed approach, for 
example for allowing regions to overlap. 

— Last, but not least, the links of hierarchical segmen- 
tation with connective segmentation [5] have to be 
studied. 
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